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A simulation of a mathematical model to compute path discrepancies 
between great circle and rhumb line flight paths is presented. The model 
illustrates that the path errors depend on the latitude, the bearing, and 
the trip length of the flight. 


I . INTRODUCTION 

A mathematical model for a comparative analysis of great circle versus 
rhumb line navigation in the continental United States has been developed 
at the Avionics Engineering Center, Ohio University. A FORTRAN simulation 
of the model has been implemented on the IBM 370 computer. The simulation 
predicts pertinent navigation information for the two flight paths. The 
basis for the project, which is a part of an M*S. thesis, is to provide a 
data base for computing discrepancies between the two flight paths. This 
document briefly describes the model and discusses the implications of the 
results obtained. 


II. BACKGROUND INFORMATION 

The standard en-route navigation system used in the United States is 
the VOR/DME. VHF Omnidirectional Range (VOR) is the basis for defining the 
airways and is therefore an integral part of air traffic control proce- 
dures. Two VOR stations are used to define a radial intersection, which is 
the basis of the established Victor airways. These Victor airways follow a 
flight path which maintains a path of constant heading (i.e. the course crosses 
the meridians at the same angle). Thus, a rhumb line course appears as a 
straight line on a Mercator projection. 

Area/Random Navigation (RNAV) as defined in FAA Advisory Circular 90- 

45A (1) is 

’...a method of navigation that permits aircraft operations on 
any desired course within the coverage of station referenced 
navigation signals or within the limits of self-contained 
system capabilities.’ 

The principal advantage of RNAV is that it allows the navigator to fly 
a great circle course between two points. The great circle course is the 


^Published as TM 89 at Ohio University. 
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shortest path between two points on the earth, and is formed by the inter- 
section of the plane defined by the center of the earth and the points of 
origin and destination, projected on the surface of the earth. The flight 
path is projected as a curved line on a Mercator projection and hence 
changes heading constantly. 

Loran-C, OMEGA, GPS, Doppler, and INS are considered RNAV systems. 
However, VOR in its basic form is not an RNAV system. A major concern is 
that the discrepancies between the two flight paths may cause navigation 
conflicts. This report attempts to quantize the discrepancies which depend 
on various factors. 


III. PATH DISCREPANCIES 

The magnitude of path discrepancies between the great circle and rhumb 
line courses between two points increases as 

1) the latitude increases, 

2) the difference in their latitudes decreases and 

3) the difference in their longitudes increases. 

When plotted on a Mercator projection map, the two flight paths start 
to diverge from the origin of the flight. The discrepancy is a maximum at 
the mid-point of the flight, and then the flight paths start to converge 
and meet at the destination point. The great circle path is always curved 
away from the equator and therefore intersects the meridians at higher 
latitudes than the rhumb line path. Also, the meridians converge as they 
depart from the equator and are closer together at higher latitudes. Thus, 
the deviation between the two paths increase at higher latitudes because 
the rhumb line path must curve rapidly to be able to fly a course of con- 
stant meridian crossing angle. 

The bearing of the flight path, which is a simple function of the lati- 
tude and longitude, also has a considerable effect on the discrepancies 
between the two flight paths. By definition, all lines of longitude are 
also great circles. On a true north-south flight path the great circle and 
rhumb line courses are exactly the same. However, as the bearing becomes 
more easterly or westerly, errors between great circle and rhumb line paths 
start to increase. The maximum path errors due to bearing alone are on a 
true east-west course. It is important to note that on an east-west course 
at the equator the great circle and rhumb line flight paths are the same 
because the equator is also a great circle -path. 

Finally, another factor that has an effect on path discrepancy is the 
trip length, which is also a function of the latitude and longitude of the 
two points. As mentioned earlier, the great circle and rhumb line courses 
start to diverge from the point of origin of the flight and reach the point 
of maximum divergence half way into the flight. Therefore, as the trip 
length increases the paths simply diverge further till they reach the half- 
way point. 
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IV. THE MODEL 


The mathematical model which was developed using the haversines formula 
was obtained from Bowditch (2). The equations developed for the model were 
designed for computer applications. The model is based upon a spherical 
approximation of the earth and adjusted using geodesic error equations for 
the North American datum based on the Clarke ellipsoid. The equations 
describing the model are given below. 


^ = Latitude of origin in degrees 

= Longitude of origin in degrees 

<|>2 = Latitude of destination in degrees 

X 2 = Longitude of destination in degrees 

^ _ (<t>j + < t > 2^ mid-latitude point 

2 

Dio = *2 "* ^1 difference in longitude 

Dlox = Interval of longitude measured from point of departure in degrees 
A. RHUMB LINE EQUATIONS 

Rhumb line bearing (relative from origin to destination) 

_ _ -1 f *2 ~ 
a Tan ( Dlo’Cos<j, ) 


Latitude of points on a rhumb line path 


= (j>j + Dlox* Cos <j>* Tana 


Rhumb line distance 


RLDIST 


60 


\ [($2 


4 > x ) 2 + (Dio ’Cosij)) 2 ] 


B. GREAT CIRCLE EQUATIONS 


Initial course of the great circle path 


C 


Tan 


-1 


Sin(Dlo) 

Cos<{>i *Tan<f »2 - Sin<^ *Cos(Dlo) 


Vertex 

The vertex is the point of highest latitude on a great circle path 


Lv = Cos 1 [Cos( <f>^) *Sin(C) ] 


Difference in longitude between vertex and point of origin 


Dlov = Sin 


r Cos(C) i 
L Sin(Lv) J 


Latitudes of points on great circle path 


^(GC) = ^ an ^ [Tan(Lv) *Cos(Dlox - Dlov)] 


Great circle distance 


GCDIST = 60'Cos' 1 [Sin ( $ ) • Sin(<j) 2 ) + Cos^) *Cos(^ 2 ) .Cos(Dlo) ] 
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C. GEODESIC ERROR EQUATIONS 

The geodesic error equations between the model and Clarke ellipsoid 

ERROR( east) = [9 . 12951 ‘CosQ - 2.92495*Cos(37)] *(X 2 - Aj)*-— 

ERROR( north) = 0. 37414 • ( $ - L ) ‘ ( jfg) “ 8.88543* [Sin(2<|, 2 ) - Sin( 2^ L ) ] 


Similar equations were also used by Hogle, Markin and Toth in their report 
Evaluation of Various Navigation Concepts 1 (3). 


V. RESULTS AND CONCLUSIONS 

The equations detailed in the previous section were modeled in FORTRAN 
on the Ohio University IBM 370 computer. Figures 1 and 2 are simulations 
of great circle and rhumb line paths on a mercator projection. The results 
indicate that the discrepancy between the two flight paths can be signifi- 
cant, Figure 1 is an east-west flight path from Baltiraore-Washington Interna- 
tional to Los Angeles International airport. At the point of maximum deviation 
the path discrepancy is 126 nautical miles for a trip length of 2017 nmi. On 
the other hand, a north-south flight path from St, Paul to Houston the discre- 
pancy at the point of maximum deviation is only 5,25 nmi for a trip length of 
903 nmi, (Figure 2,) 

Since shorter trip lengths are of major importance to the pilot an 
effort was made to quantify the path errors for flight paths of up to 500 
nmi. Figures 3 through 6 illustrate how the factors mentioned earlier 
effect the path discrepancies. The flights were simulated at a constant 
bearing with the mid-point of the flight at the specified latitude. Com- 
parisons of the plots indicate the compound effect bearing and latitude of 
the mid-point of a flight have on the path errors. A flight at a mid-point 
latitude of 35 degrees north and a relative bearing of 76 degrees has maxi- 
mum path errors of 1.1 nmi for trip lengths of up to 300 nmi (Fig. 3). 

However, a flight at the same mid-point latitude, but a relative bearing of 
39 degrees has maximum path errors of more than 6 nmi for trip lengths up 
to 450 nmi (Fig. 5). 

It is of vital importance to note here that the errors mentioned in 
this report are solely due to the discrepancies in the great circle and 
rhumb line paths. The errors do not account for reciever computational 
errors or pilot errors. Another source of error may be discrepancies among 
the earth models in various RNAV recievers. Offsets due to using different 
navigation systems in the same airspace also may cause considerable dis- 
parity (4). It is recommended that any decision made regarding this subject 
must take into account all discrepancies mentioned above, and not just the 
goemetrical errors. 
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